A global navigation satellite system (GNSS), such as the Global Positioning System (GPS), can be used to provide navigation information, e.g., position and velocity measurements for a sensor platform such as a vehicle, robot, or handheld device. When a GNSS is not available, an inertial navigation system (INS) can be used to provide the position and velocity measurements or estimates by using inertial sensors and an integration algorithm. Pure strap down inertial navigation has drift, that is, the errors in estimated position increase with time. A monocular camera can be used as an aiding source to address the drift.
For monocular camera aided navigation a system can extract features from a set of two or more frames (e.g., images) and identify matching features (also referred to herein as “feature matches”) in the set. These feature matches can be used to, for example, determine relative motion between the images, identify a moving object in the images, and determine a relative location of two or more cameras with respect to objects in the images.
In image based navigation, a camera associated with (e.g., mounted on) an aircraft or ground vehicle can serve as an aiding sensor for a navigation system (e.g., an inertial navigation system). The camera can track stationary landmarks on the ground and based on a determination of its own motion, an integrated navigation system can estimate where it should see landmarks in the next camera frame. The camera can track features through feature matching between two frames or between a map and a new frame. The locations of matched features can be fed to a Kalman filter along with the position, velocity, and attitudes calculated by the strapdown INS to perform the correction of drift errors mentioned above. The Kalman filter fuses information from the INS and the camera to generate optimal estimates of the state of the sensor platform at each point in time.